The techniques of face detection and face recognition are each being explored by those skilled and a great many advances have been made in those respective fields in recent years. Face detection has to do with the problem of locating regions within a digital image or video sequence which have a high probability of representing a human face. Face recognition involves the analysis of such a “face region” and its comparison with a database of known faces to determine if the unknown “face region” is sufficiently similar to any of the known faces to represent a high probability match. The related field of tracking involves face or identity recognition between different frames in a temporal sequence of frames. A useful review of face detection is provided by Yang et al., in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 1, pages 34-58, January 2002. A review of face recognition techniques is given in Zhang et al., Proceedings of the IEEE, Vol. 85, No. 9, pages 1423-1435, September 1997.
Face tracking for digital image acquisition devices includes methods of marking human faces in a series of images such as a video stream or a camera preview. Face tracking can be used for indication to the photographer the locations of faces in an image, improving the acquisition parameters, or for allowing post processing of the images based on knowledge of the location of faces.
In general, face tracking systems employ two principle modules: (i) a detection module for location of new candidate face regions in an acquired image or a sequence of images; and (ii) a tracking module for confirmed face regions.
A well-known fast-face detection algorithm is disclosed in US 2002/0102024 and at Rapid Object Detection Using a Boosted Cascade of Simple Features, in Proc. IEEE Conf. on Computer Vision & Pattern Recognition, 2001; (describing Haar-feature detection techniques). In brief, Viola-Jones first derives an integral image from an acquired image—usually an image frame in a video stream. Each element of the integral image is calculated as the sum of intensities of all points above and to the left of the point in the image. The total intensity of any sub-window in an image can then be derived by subtracting the integral image value for the top left point of the sub-window from the integral image value for the bottom right point of the sub-window. Also intensities for adjacent sub-windows can be efficiently compared using particular combinations of integral image values from points of the sub-windows.
In Viola-Jones, a chain (cascade) of 32 classifiers based on rectangular (and increasingly refined) Haar features are used with the integral image by applying the classifiers to a sub-window within the integral image. For a complete analysis of an acquired image this sub-window is shifted incrementally across the integral image until the entire image has been covered.
In addition to moving the sub-window across the entire integral image, the sub window must also be scaled up/down to cover the possible range of face sizes. In Violla-Jones, a scaling factor of 1.25 is used and, typically, a range of about 10-12 different scales are required to cover the possible face sizes in an XVGA size image.
It will therefore be seen that the resolution of the integral image is determined by the smallest sized classifier sub-window, i.e. the smallest size face to be detected, as larger sized sub-windows can use intermediate points within the integral image for their calculations.
A number of variants of the original Viola-Jones algorithm are known in the literature. These generally employ rectangular, Haar feature classifiers and use the integral image techniques of Viola-Jones.
Even though Viola-Jones is significantly faster than other face detectors, it still requires significant computation and, on a Pentium class computer can just about achieve real-time performance. In a resource-restricted embedded system, such as hand held image acquisition devices (examples include digital cameras, hand-held computers or cellular phones equipped with cameras), it is not practical to run such a face detector at real-time frame rates for video. From tests within a typical digital camera, it is only possible to achieve complete coverage of all 10-12 sub-window scales with a 3-4 classifier cascade. This allows some level of initial face detection to be achieved, but with unacceptably high false positive rates.
Census transform techniques are described at Froba, B. and Ernst, A., Face detection with the modified census transform, in Automatic Face and Gesture Recognition, 2004; Sixth IEEE International Conference on, 17-19 May 2004 Page(s): 91-96 of Proceedings.
Soft cascade techniques and use of cumulative probabilities are described at Bourdev, L. and Brandt, J., Robust object detection via soft cascade, in Computer Vision and Pattern Recognition, 2005 (CVPR 2005). IEEE Computer Society Conference on, Volume 2, Issue, 20-25 Jun. 2005 Page(s): 236-243 vol. 2.
Use of Haar-like filter for face recognition is described at Y. Higashijima, S. Takano and K. Niijima. Face recognition using long Haar-like filters, in Proceedings of the Image and Vision Computing New Zealand 2005 (IVCNZ2005), pp. 43-48, 2005.
The above-cited references, as well as all references cited below, and the background and brief description of the drawings section, and the drawings, are hereby incorporated by reference into the detailed description as providing alternative embodiments. In addition, U.S. Pat. Nos. 7,620,218, 7,606,417, 7,315,631, 7,469,071, 7,403,643, 7,362,368, 7,551,755, 7,558,408, 7,587,068, 7,555,148, 7,564,994, 7,317,815, 7,269,292, 7,315,630, 7,460,694, 7,466,866, 7,460,695 and 7,440,593; and United States published patent applications nos. 2009/0273685, 2009/0238419, 2009/0263022, 2008/0220750, 2009/0244296, 2009/0190803, 2009/0189998, 2009/0052750, 2009/0185753, 2009/0196466, 2009/0080797, 2009/0080713, 2008/0316328, 2008/0266419, 2008/0037840, 2008/0220750, 2008/0219581, 2008/0037839, 2008/0037827, 2008/0175481, 2008/0043122, 2007/0269108, 2007/0147820, 2006/0285754 and 2006/0204110; and United States patent applications serial nos. PCT/US2006/021393, Ser. Nos. 12/512,796, 12/374,020, 12/572,930, 12/191,304, 12/485,316, 12/479,593 are hereby incorporated by reference.